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In  ^hia  paper  ^.jfepresenl^a  subgradient  algorithm  for  the  problem 

Min  ^F(x);  x^C'^y  vhere  F(x)  ■ Max  ^f^(x);  1 “ 1,  2,  ....  mV  and  where 

M w It.  (»  t 

f.(x)  " (jaIx) . Each  f..  la  assumed  to  be  a proper  convex  function,  and 


the  number  of  different  subgradient  sets  associated  with  nondlfferentlable 
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points  of  Is  sssuasd  to  be  finite  on  any  bounded  set.  P^obl'^ms  belonging 

to  this  class  Include  those  where  the  f .L  are  % ^ - noms  (1  < p < »);  for 

IJ  P ~ ~ 

example,  the  linear  approximation  problem  and  both  the  mlnlmax  and  mlnlsum 
problems  of  location  theory.  >>  The  algorithm  Is  an  extension  of  the  work  of 


Dem'yanov  and 

We  prove  convergence  of  the  algorithm  to  an  e - optimal  solution^  and 
f" demonstrateJ^ts  effectlvenesgl  by  solving  a number  of  problems  from  location 
theory  and  linear  approximation  theory.  Our  computational  results  are  compared 
with  other  solution  methods. 
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Abstract 

In  this  paper,  we  present  a subgradient  algorithm  for  the  problem 

Mlu  {F(x);  X € where  F(x)  - Max  {f^(x);  1 > 1,  2,  . . . , m}  and  where 

f (x)  - ^ f.  Ax).  Each  f.  Is  assumed  to  be  a proper  convex  function,  and 

the  number  of  different  subgradient  sets  associated  with  nondlfferentlable 

points  of  f^j  Is  assumed  to  be  finite  on  any  bounded  set.  Problems  belonging 

to  this  class  Include  those  where  the  f..  are  I - norms  (1  < p < "■);  for 

IJ  p - - 

exastple,  the  linear  approximation  problem  and  both  the  mlnlmax  and  mlnlsum 
problems  of  location  theory.  The  algorithm  Is  an  extension  of  the  work  of 
Dem'yanov  and  MalozemOV  [6]. 

We  prove  convergence  of  the  algo:.‘lthm  to  an  c - optimal  solution  and 
demonstrate  Its  effectiveness  by  solving  a number  of  problems  from  location 
theory  and  linear  approximation  theory.  Our  computational  results  are  compared 
with  other  solution  methods. 
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1 . Introduction 


The  tolnlmax  problem  that  we  consider 

(P)  Min  F(x),  F(x)  - Max  {f  (x);  1 - 1,  2 m}, 

xcR 

I 

f . ■ ^ f..,  f..  convex  not  necessarily  differentiable, 

1 j.i 

Is  a special  case  of  an  unconstrained  nondlfferentlable  convex  programming 
problem.  For  m ~ 1,  we  minimize  a sum  of  convex  nondlfferentlable  functions 
and  we  call  this  problem  a mlnlsum  problem. 

To  solve  such  nondlfferentlable  convex  problems,  It  seems  straightforward 
to  extend  the  steepest  descent  method,  using  the  fact  that  the  steepest  descent 
direction  Is  given  by  the  opposite  of  the  element  of  minimum  norm  In  the  sub- 
gradient set.  Unfortunately,  it  Is  well-known  that  this  extended  procedure 
Is  In  general  not  convergent  [6,  19l-  As  is  Indicated  in  [4],  it  Is  necessary 
to  consider  a larger  set  than  the  subgradient  set  Itself  In  order  to  guarantee 
convergence . 

The  methods  proposed  In  the  literature  to  accomplish  this  can  be  divided 
Into  two  main  families. 

The  first  family  enlarges  the  subgradient  set  at  any  point  and  determines 
a descent  direction  from  this  enlarged  set.  For  example,  Dem'yanov  and 
Malozemov  [6]  solve  mlnlmax  problems  with  continuously  differentiable  functions 
by  considering  also  'near  binding*  functions  In  the  following  way:  to  the 
extreme  points  of  the  subgradient  set  (formed  by  the  gradients  of  the  binding 
functions)  they  add  the  gradients  of  those  functions  which  are  almost  blndiiig. 

In  another  study,  Bertsekas  and  Mltter  [2I  used  the  c-aubgradlent  set  to 
calculate  a descent  direction.  All  of  these  methods  require  the  knowledge 
of  the  complete  subgredlent  set. 

By  contrast,  the  second  family  requires  only  partial  local  Information, 

In  general  only  one  element  of  the  eubgradlent  set.  At  each  step,  a aubgradient 
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is  added  to  a 'bundle'  of  previous  subgradients  and  a descent  direction  Is 
obtained  from  this  bundle.  At  some  steps,  the  bundle  Is  reinitialized  so 
that  It  always  consists  of  a limited  nusiber  of  subgradients.  This  Idea  has 

t 

been  proposed  Initially  by  Lemarechal  [14]  and  Wolfe  [19].  A nice  feature 
of  these  methods  Is  that  they  are  conjugate  gradient  methods  [15]  when 
applied  to  smooth  problems.  However,  It  Is  well-known  that  conjugate 
gradient  methods  require  very  accurate  line  searches  [11,  15],  thus  con- 
siderable time  has  to  be  spent  on  the  line  search  part  of  these  algorithms. 
Further  extensions  of  these  methods  have  been  developed  by  Feuer  [9]  and 
recently  by  Mifflin  [16].  In  these  two  works,  the  Idea  of  bundling  Is 
Intimately  related  to  the  generalized  gradient  of  Clarke  [3]. 

Our  approach  Is  more  related  to  the  first  class  of  methods  In  that  we 
require  the  same  type  of  local  Information.  However,  we  Introduce  the  concept 
of  considering  subgradients  at  neighboring  points  of  the  current  Iterate.  Thus 
In  some  sense,  we  anticipate  nondlfferentlablllty  [4,  p.  43]. 

As  an  overview  of  what  follows.  In  Section  2 we  Introduce  notation  and 
definitions.  In  Section  3,  we  discuss  the  outcome  of  the  direction  finding 
optimization  problem  and  show  that  we  either  find  a direction  of  descent, 
or  we  have  attained  a near  optimal  point.  Then,  after  presenting  the  algorithm 
In  Section  4,  we  prove  In  Section  5 that  It  Is  convergent.  In  this  section, 
we  combine  the  methods  used  In  [4]  and  [6]  but  Is  Is  also  necessary  to  rely 
on  the  properties  of  the  functions  Involved.  Section  6 contains  details  for 
a modified  version  of  the  algorithm,  while  Section  7 gives  some  computational 
results.  The  following  notations  and  basic  definitions  will  be  used  In  the 
remaining  sections. 
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We  denote  n>dlmenalonal  Euclidean  space  as  R°  and  ||x||p  as  the  £p-norm 
of  X e where  | |x|  ( is  the  Given  a point  x e R°,  we  denote  the 

Euclidean  ball  about  x of  radius  n as  N(x,  n).  In  the  case  where  x - 0 and 
n “ 1|  B = N(0,  1);  that  is,  B is  the  Euclidean  unit  ball. 

Given  a function  F,  defined  on  r“,  the  subgradient  set  of  F at  x is 
denoted  as  3F(x),  and  the  directional  derivative  of  F at  x in  the  direction 
y is  F'(x,  y). 

Given  a subset  S c r",  we  let  Conv(S)  be  the  convex  hull  of  S and  Nr(S) 
be  the  element  of  soinimuiD  Euclidean  norm  in  S when  S is  closed  and  convex. 
Given  any  function  F defined  on  S,  we  let  3F(S)  = u {3F(x);  x € S}. 
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2.  Definitions  and  Notations 


We  shall  assume  that  the  functions  f 


ij 


Involved  In  the  definition  of 


F(x)  - Max  {f.(x)  - i f..(x);  1 * 1,  2,  ....  m} 
j-1  ^ 

are  convex  and  finite  on  r".  For  notatlonal  convenience,  we  write  each  f 


as  the  sum  of  exactly  I functions  f^^,  assuming,  If  necessary,  that  some  of 
the  f^j  are  Identically  zero  on  r”.  Then  It  Is  clear  that  each  f^  Is  a con- 
tinuous convex  function  and  thus,  F Is  a finite  convex,  continuous  function  on 

r". 


Suppose  e ^ 0 Is  given.  At  each  point  x,  we  consider  the  set  of  Indices 


R(x, 

e)  : 

i {1 

€ 

{1,  2,  ..., 

m};  fj^(x)  ^ 

F(x)  - e}. 

(2, 

.1) 

In  particular 

R(x, 

0) 

- {1 

e 

{Ip  2p 

m};  fj^(x)  - 

F(x)},  and 

(2. 

.2) 

R(x, 

^1^ 

c R{x, 

^2^ 

0 £ e 

I- '2- 

(2, 

.3) 

We  shall  make 

use 

of 

the  following  property 

tl?!: 

3F(x) 

- 

Conv 

(U 

{3f^(x);  i 

c R(x,  0)}). 

(2. 

,4) 

We  restrict  our  attention  to  the  functions  f^^  which  belong  to  the  class 
of  functions  defined  below. 

Definition  2.1  A finite  convex  function  Is  LFS  If,  In  any  closed  bounded 
Euclidean  ball,  the  number  of  different  subgradient  sets,  corresponding  to 
the  points  of  nondlffarentlablllty  of  this  function.  Is  finite.  (LFS  Is  an 
abbreviation  for  the  phrase ' locally  finitely  subdlf ferentlable* ) . 

Assumption  A.l  The  functions  are  LFS. 

The  concept  of  LFS  functions  originated  from  the  study  of  minimization  pro- 
blems In  location  theory,  where  functions  Involving  norms  are  frequently  en- 
countered, such  as  I |x  - slip,  1 < p < «.  The  function  ||*llj^  (for  * « 1® 

not  differentiable  along  either  axis,  but  the  total  number  of  different  sub- 
gradient sets  associated  with  Sll  points  of  nondlfferentlablllty  Is  five. 
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As  an  example,  we  note  that  piecewise  linear  functions  are  also  LFS. 


The  representation  of  each  function  f^  as  a sum  of  LFS  functions  may 

be  useful,  because  the  sum  of  LFS  functions  Is  not  always  LFS.  As  an  example, 

2 2 2 

consider  x^^  *2  N*l  lj^»  * “ *2^  ^ ^ ’ which  viewed  as  a single 

2 2 

function  is  not  LFS.  But  f + *2  ^2  - I 

In  the  next  definition,  we  introduce  the  points  of  nondlff erentlablllty 
of  a function  ^ • 

Definition  2.2 

Gj^j(x,  n)  = {x}  u {y;  y e N(x,  n),  fj,j  not  differentiable  at  y}. 

We  then  define  the  set  S,  which  Is  our  enlarged  subgradient  set,  obtained 
by  considering  also  neighboring  points  of  nondlff erentlablllty. 

Definition  2.3 

I 

(x,  n)  = I 3f..  (G.  (x,  n)),  i - 1,  2 (2.5) 

J-1  J J 

S (x,  e,  n)  = Conv'  (u  {S^  (x,  n);  1 c R (x,  e)}).  (2.6) 

The  following  example  Illustrates  Definition  2.3. 

Example  1.  Consider  the  problem 

Mln2  Max  {fj^(x),  f2(x)},  fj^(x)  - ||x|lj^  - |xj^|  + 1x2 1, 

X€  R 

f2(x)  - ||x  - (1,  -1)  llj^  - |xj^  - l|  + 1*2  ^1* 

Let  ■ 1,  e » .2  and  n - 1.  At  Xq  ■ (.9,  1)  we  have 
^iCxq)  ■ 1.9, 

3f^(xQ)  - {(1,1)},  3f2(Xo)  " !)}• 

Furthermore,  In  N(Xq,  n),  there  are  points  of  nondlfferentlabllity 

(a)  for  f^,  along  the  axis  0 and  at  (.9,  0),  and 

(b)  for  f2,  along  the  line  Xj^  ■ 1. 

These  supplementary  points  yield  two  more  different  sets  to  Include  in  S, 

Conv  ({(1,  -1),  (1,  1)}),  Conv  ({(-1,  1),  (1,  1)}). 
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Hence,  S(Xq,  e,  n)  is  the  triangle 

Conv  ({(-1,  1),  (1,  1),  (1,  -1)}). 

On  the  other  hand,  suppose  we  choose  A =•  2 with  ” 1*2^1 » " l*2l’ 

f2i(x)  “ [x^  - l|,  £22^  ” 1*2  + l|*  case,  it  is  easy  to  check  that 

Sj^(Xq,  n)  = Conv  ({(1,  1),  (-1,  1),  (-1,  -1),  (1,  -1)}), 

^2^*0’  (Ul,  1),  (-1.  I)}). 

Then  ^^*0'  square 

S(xg,  e,  n)  =•  Conv  ({(1,  1),  (-1,  1),  (-1,  -1),  (1*  -1)}). 

Remark . From  Definitions  2.2  and  2.3,  it  is  clear  that  3f^, (x)  c 3f ^ (G^^. (x,  n)). 


j ” 1>  2,  *>•,  ^ t i 1>  2,  ...,m, 
and , 

i I 

3f,(x)  = I 3f.,(x)  c I 3f  (G.  (x,  n))  - S.(x,  n), 

^ j=l  j-1 

1 * 1,  2,  ••.,  m.  (2.7) 

Since,  by  (2.3), 

R(x,  0)  c R(x,  e), 
we  get  from  (2.7) 

u {3f^(x);  i e R(x,  0)}  c u {Sj^(x,  n);  1 £ R(x,  0)} 

c u {S^(x,  n);  i € R(x,  e)}, 

and  consequently, 

Conv  ( u {3f^(x);  1 € R(x,  0)})  c Conv  ( u {S^(x,  n);  1 £ R(x,  e)}).  (2.8) 

Then  (2.4)  and  (2.8)  imply 

3F(x)  c s(x,  e,  n).  (2.9) 

From  Definition  2.3,  it  is  easy  to  establish  the  following  property. 

Property  2.1.  S(x,  e,  n)  is  a nonempty  compact  convex  subset  of  R**. 
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t^root . f rom  Assumption  A.l,  each  set 


ij  J 

Is  a finite  union  of  nonempty  compact  sets,  so  that  3f (x,  n))  is  Itself 
a compact  set.  Hence,  the  finite  sum 
i 

I n))  - S^(x,  n),  i - 1,  2,  ...,  m, 

is  also  a compact  set  and  the  set 

u {Sj^(x,  n);  i e R(x,  e)} 

is  a finite  union  of  nonempty  compact  sets.  Thus,  this  set  Is  also  a nonempty 
compact  set  and  Property  2.1  Is  a direct  consequence  of  Theorem  17.2  of  [18].  || 

S(x,  e,  n),  which  contains  the  subgradient  set  but  also  pertinent  local 
information,  now  replaces  this  subgradient  set  for  the  determination  of  a 
descent  direction  and  the  corresponding  directional  derivative,  provided 
that  they  exist.  These  will  be  given  by  the  next  definition. 

Definition  2.4.  At  any  point  x,  let 

i>(x,  e,  n)  = Min  ]Max  {(g,  d);  d € S(x,  e,  n)  ; Hgjl  < 1).  (2.10) 

• > 

By  compactness  of  S{x,  e,  n)  and  of  the  Euclidean  unit  ball  B,  this 
quantity  is  always  well-defined.  Using  a mlnimax  theorem  (Corollary  37.3.2,(18]) 
since  the  two  sets  S(x,  e,  n)  and  B are  both  non-empty  compact  convex  sets, 
we  obtain 
Property  2.2 

e,  n)  - -||Nr(S(x,  c,  n))||.  (2.11) 

Proof.  i(<(x,  E,  n)  “ Min  {Max  {(g,  d);  d € S(x,  e,  n));  g e B} 

■ Max  (Min  {(g,  d) ; g e B};  d e S(x,  e,  n)} 

- Max  {(d,  -d/||d||);  d e S(x,  e,  n)) 

- - Min  {j|d||;  d c S(x,  e,  n)) 

■ -|  |llr(S(x,.  e,  n))  1 1 . I i 

As  a consequence  of  Property  2.2,  we  have  i|)  > 0 or  i|)  < 0.  These  are  two 
cases  that  we  consider  in  the  next  section. 
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3.  Study  of  the  Alternative  i|<(x,  e,  n)  ■ 0 or  i|)(x,  e,  n)  < 0 

As  shown  In  Section  2,  c,  n)  Is  similar  to  a directional 

derivative.  Hence  it  is  reasonable  to  expect  that  its  value  will  give  some 
clue  about  the  optimality  of  the  point  x.  Before  considering  the  two  cases 
e,  n)  =0  and  ^fCx,  t,  n)  < 0,  we  prove  two  lemmas  which  will  be  useful 
in  the  sequel. 

Lemma  3 . 1 Let  X be  a non-empty  compact  subset  of  k".  Then  the  subgradients 
of  the  functions  f^j  are  uniformly  bounded  on  the  set  X'  ■ {y;  y f N(x,  n) , 

X £ X}.  That  is,  0 < M < “ such  that 

given  any  y £ X'  and  any  s c 

ll®M  ^ •••»  ®i  “l*  2, 

Proof . Since  X is  compact,  X'  is  also  compact.  Then,  by  Theorem  24.7  of 
[18],  3f^j(X')  is  a non-empty  compact  subset  of  r"  and  consequently,  the  number 
M^j  = Sup  {|ls||;  s £af^j(X’)} 
is  finite.  Then 

M = Max  1 1,  2 j • 1,  2,  t} 

satisfies  the  requirements  of  the  Lemma.  | | 

In  Lemma  3.2,  we  shall  consider  the  following  piecewise  linear  function: 
V(z)  = Max  z)  - k - 1,  2,  ...q}, 

in  which  the  Wj^  are  q given  vectors  of  R*'  and  the  Vj^  are  q real  numbers. 

Lemma  3.2  A necessary  and  sufficient  condition  that  the  inf imum  v*  of  V(z) 
be  attained  is  that  there  exists  a convex  combination  of  the  w^  equal  to  the 
null  vector,  that  is,  k - 1,  2,  ...,  q,  > 0 such  that 

^ Xj^  - 1 and  J Xj^wj^  - 0. 

Furthermore,  v*  > - ^ ^k'^k  ^k  unique, 

k 

V*  • - I X.  V . 
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Proof.  The  Infimum  of  V(z)  Is  attained  if  and  only  if  the  linear  program 
Min  V 

s.t.  (Wj^,  z)  - Vj^  - V < 6 k - 1,  2,  ....  q 
has  an  optimal  solution.  Its  dual  may  be  written 
Min  I 


^ \\  ° ° 

k 

I \ = 1 

k 

k“l,  2,  ...,q. 

By  duality  theory,  if  the  dual  is  not  feasible,  v*  - Otherwise,  v*  is 

finite  and  for  any  dual  feasible  solution  k • 1,  2,  ...,  q» 

V*  1 - I W . 

k 

If  there  is  only  one  feasible  solution,  then 

V*  - - [ A V I I 


We  now  consider  ili(x,  e,  n)  " 0.  We  call  such  a point  x,  at  which 
ij((x,  c,  n)  • 0,  a stationary  point.  Let  F*  ■ F(x*)  be  the  function  value 
at  an  optimal  point  x*,  provided  such  a point  exists.  From  the  two  previous 
Jemmas,  we  obtain  the  following  theorem  which  provides  a bound  on  the  differ- 


ence between  F*  and  the  function  value  at  a stationary  point. 

Theorem  3.3  For  any  stationary  point  x of  a compact  set  X,  we  have 
F(x)  -e-2*nM<F*<  F(x>. 

Proof . Statlonarlty  at  x Is  equivalent  to  0 e S(x,  c,  n). 


(3.1) 


With  S(x,  E,  n)  as  the  convex  hull  of  u {S^(x,  n) ; 1 c R(x,  e)},  0 c S(x,  e,  n) 
if  and  only  if  there  exists  such  that 

, k - 1,  . . . , q < n + 1,  (3.2) 

(3.3) 

K.  X 

where. 


X,  > 0 
k - 


Wk  e u {Sj^(x,  n);  i e R(x,  e)} 


k-1 


(3.4) 


and 


k-1 


^k'^k  * °' 


(3.5) 


For 


arbitrary  k,  1 < k < q»  we  have  that  w.  e S (x,  n)  for  some  h e R(x,  e).  But 
- — K n 


then  from  the  definition  of  Sjj(x,  n) , we  have  that  Wj^  - ^ where 

r»l 

Let  yj^^  e '^kr  ® ^^hr^^kr^  for  r - 1,  2,  ....  1. 

From  the  subgradient  inequality,  Vr>l,  2,  ...,  I, 

^hr^"^  i ^hr^^kr^  * " ^kr^  ‘ 

Furthermore  for  arbitrary  s.  e 3f.  (x), 

iir  Mr 


(3.6) 


(3.7) 


(3.8) 


fhr<ykr>  i ^hr^*>  + ^®hr'  ykr  " *>' 

Adding  (3.7)  and  (3.8)  yields 

^hr^*>  i ^hr^*>  + * - ykr>  + ^®hr'  ^kr  " *>* 

With  X £ X,  a compact  set,  from  Lessta  3.1,  there  exists  M such  that  M > 11*1,^11 
and  thus, 

^®hr*  ^kr  " *)  i " '’**•  (3.10) 
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Using  (3.10)  and  summing  (3.9)  over  all  r - 1, 


• • • » ^ » 


i I I 

I ^ - AnM  + [ (w  z - y ), 

-1  r-1  " r-1 


or 


i 

- tnM  + ^ (Wj^^, 

r-1 


^ - ykr>- 


Now,  since  h e R(x,  e)  , fj,(x)  > F(x)  - e and  then  from  (3.11) 

i 

F(z)  > ff^(2)  > F(x)  - e - «nM  + (Wj^^,  z - yj^^), 

r-1 


(3.11) 


or 


F(z)  > F(x)  - e - *nM  + I (Wj^^,  z - y^^) . 


r-1 


^kr' 


(3.12) 


Since  k is  arbitrary,  we  obtain 


F(z)  > F(x)  - e - InM  + Max  (wj^^,  z - y^^) ; k - 1 q}.(3.13) 


r-1 


'kr' 


Letting  (3.13),  noting  (3.2),  (3.4)  and  (3.5)  and  taking 

r-1 

the  minimum  over  z e on  both  sides  of  (3.13)  we  have  by  Lemna  3.2, 

F*  = Min  F(z)  > F(x)  - e - XnM  - f X v.  . (3.14) 

k-1  ^ ^ 
i 

Decomposing  \ • I (w,  , y.  ) Into 
r-1 

I I 

\ ^kr  -*)  + I (V 

r*l  r*l 

and  using  the  Inequality 

i 'V-  V -»)■'■' «. 

we  get 

I " ^j^^''kr’  “ ^’'k' 
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► 


But  then, 


> - ^ X.  UnM  - f *)• 


k-1  k-1  k-1 

Using  (3.4),  (3.5)  in  (3.15)  we  get 


- ? ^uv.  > - AnM. 


Therefore  (3.14)  and  (3.16)  give 
F*  > F(x)  - e - 2JlnM. 


(3.15) 


(3.16) 


(3.17) 


We  now  present  an  example  which  Illustrates  Theorem  3.3. 

Example  2.  Consider  the  very  simple  problem 

Mln2F(x),  F(x)  - 1 |x|  - |xj^|  + |x2|. 

xeR^ 

With  c - 0 and  n ~ 1,  we  look  at  two  different  formulations  of  this  problem. 

(a)  F(x)  » I |x|  with  m » 1,  1-1  and  clearly  M - vT.  The  point  Xq  - (1,  1) 

is  stationary  since  S(Xq,  c,  n)  is  the  convex  hull  of  the  three  points 

(-1,  1),  (1,  1),  (1,  -1). 

At  Xq,  we  have  F(Xq)  - 2 and  since  P*  - 0,  we  verify 
-0.82  - 2 - 2 s rT  < 0 < 2. 

(b)  F(x)  - jxj^l  + IX2I  with  m - 1,  1-2  and  M - 1.  Xq  is  also  stationary 
because  S(Xq,  c,  n)  is  the  square 

Conv  ({(1,  1),  (-1,  1),  (-1,  -1),  (1,  -1)}). 

Then,  Theorem  3.3  yields 

-2  - 2 - (2) (2)  < 0 < 2. 

This  example  Illustrates  a general  result.  VAien  each  function  f^  Is  Itself 
LFS,  choosing  1-1  will  give  better  bounds  In  the  Inequalities 
F(x)  - e - 21nM  < P*  < P(x). 
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To  see  this  result,  write  f as  and  decompose  any  subgradient  s^^  of 

r-1 

at  X into  the  sum 


s^  - 8^^  e 3fj^y(x),  r - 1,  ....  1. 

r-1 


(3.18) 


If  we  call  M the  bound  on  the  norm  of  any  subgradient  of  any  function  f^^ 
and  the  bound  for  the  subgradients  of  the  functions  f^,  1 ■ 1,  . . . , m, 
from  (3.18)  we  obtain 


IsJI  - III  < IM 

r»l 


and  consequently 


< iM. 


(3.19) 


Writing  now  the  inequalities  given  by  Theorem  3.3  in  both  cases,  i > 1 and  i . 1, 


we  have 

T(x)  - e - 2inM  < F*  < F(x),  (3.20) 

F(x)  - e - 2nMj^  1 F*  < F(x).  (3.21) 

But  inequality  (3.19)  indicates  that 

F(x)  - e - 2*tiM  < F(x)  - e - 2nMj^  < F* 
so  that  in  general  (3.21)  gives  better  bounds. 

On  the  other  hand,  considering  each  f^  as  a sum  of  i > 1 LFS  functions  • 
could  ease  the  task  of  determining  the  sets  S^(x,  n).  Hence,  there  is  a trade-off 
between  accuracy  and  implementation. 

The  set  of  stationary  points  does  not  seem  to  have  any  obvious  property, 
such  as  convexity.  With  the  problem  of  Example  1,  and  choosing  1 > 1 
Min  Max  {|xj^|  + IX2I,  |x.  - l|  + |x2  + l|}, 

(x^,  *2) 

the  set  of  optimal  points  .is  the  line-segment  Joining  the  two  points 

(0,  -1)  and  (1,  0).  But,  with  e ■ .2  and  n ■ 1,  the  set  of  stationary  points  is  es 

sketched  in  Figure  1. 
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We  now  consider  the  remaining  case,  i|/(x,  e,  n)  < 0. 

Since  x is  not  stationary,  we  are  able  to  find  a descent  direction. 

Theorem  3.4  If  ij/(x,  e,  n)  < 0,  then  there  exists  a nontrivial  descent  direction 
for  F at  X. 

Proof . t(»(x,  e,  n)  < 0 implies  the  existence  of  a direction  for  which 

0 > i|;(x,  e,  n)  - -||Nr(S(x,  e,  n))M 

- Max  {(d,  gp);  d e S(x,  e,  n)>  (3.22) 

--lldoll 

- Wq.  go)- 

Thus,  the  property 

F'(3c,  gg)  “ Max  {(d,  gg);  d e 3F(x)} 
and  (2.9),  (3.22)  imply 

f'  (x.  go)  •=  0-  II 


IS 


4.  Description  of  the  Algorithm 


I Using  the  results  of  Section  3,  particularly  Theorems  3.3  and  3.4,  it  is 

r clear  how  to  construct  a descent  method  for  problem  (P) . 

I Choose  e > 0,  n > 0 and  x_  a starting  point.  Set  k ■ 0 and  go  to  Step  1. 

I 

I Step  1 

(At  x^,  find  and  e).  Calculate  S(x^,  e,  rj)  and  h). 

Go  to  Step  2. 

r 

I Step  2 

If  e.  h)  ■ 0,  stop:  Xj^  is  a stationary  point.  If  e,  n)  < 0, 

there  exists  gj^  for  which 

ij)(Xj^,  t,  n)  - Max  {(d,  gj^);  d e S(xj^,  e,  n)}. 

Line  search:  find  t,  such  that 

k 

Set  = Xj^  + k ” k + 1 and  return  to  Step  1. 

The  next  section  establishes  that  this  algorithm  converges  to  a stationary 
point  for  the  class  of  problems  studied  in  this  paper. 


i 

I 
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5.  Proof  of  Convergence 


For  the  proof  of  convergence,  we  assume  there  exists  some  such  that 
the  level  set  X - {x;  F(x)  < F(Xq)}  is  bounded.  Since  F is  continuous,  we 
deduce  that  the  minimum  value,  F*,  of  F is  attained  at  some  x*  e X. 

Now  suppose  that  the  starting  point  is  x^.  The  algorithm  generates  a 

< F(x^)  <F(Xq)  by  Theorem  3. A. 

If  the  algorithm  terminates  at  some  iteration  k,  by  Theorem  3.3,  we  have  an 
estimate  of  F*.  Otherwise,  the  Infinite  sequence  {Xj^}  must  have  a limit  point 
e X.  Let  K c {0,  1,  2,  ...}  be  the  set  of  indices  such  that  Xj^  -*■  x^ 
for  k -►  “>  and  k e K. 

We  now  show  (by  contradiction)  that  x^  is  a stationary  point,  that  is, 
ij((x^,  e,  n)  • 0. 

Assume  that  'J'Cx^,  e,  n)  “ -b  < 0.  First,  we  show  that  for  k large  enough, 
i(((Xj^,  e,  n)  is  uniformly  negative,  l.e.,  e,  n)  < c < 0.  This  follows 

because  in  a neighborhood  of  x^,  the  sets  S(Xj^,  e,  n)  approximate  S(x^,  e,  n) 
[A,  p.  43]. 

Lemma  5.1  There  exists  a number  such  that  k c K and  k > implies 
R(xj^,  e)  c R(x^,  e). 

Proof . By  Lemma  7.1  of  [6,  p.  92],  56  > 0 such  that  | |x^  ~ x| | <6  implies 
R(x,  e)  c R(x^,  e).  Since  x^  -*•  x^  for  k e K and  k -►  <»,  we  can  find  for 
which  k > Implies  | |x^  - x^||  < 6 and  consequently  e)  c R(x^,  e).  || 

Lemma  5.2  Let  g:  R°  R be  an  LFS  function  and  suppose  {xj^;  k c K}  -►  x^  is  a 

convergent  sequence.  Then,  for  any  Y ^ 0,  there  exists  N such  that 

3g(G(Xj^,  n))5  3g(G(x*,  n))+  yB,  Vk  > N,  k E K.  (5.1) 

Proof . For  any  x e R®  we  define  G' (x,  n)  = G(x,  n)  /x  so  that  G(x,  n)  ■ 
g' (x,  n)  u (x). 


sequence  of  points  {x^^}  c X,  since  F(Xj^^j^) 
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From  Corollary  24.5.1  of  [18]  there  exists  N’  such  that  Vk  > N',  k € K we  have 
3g(Xk)  c 8g(x^)  + yB.  (5.2) 

We  now  consider  G'(Xj^,  n) . Define  the  set 

H(8)  = u {3g(G'(x^,  n));  k > s,  k c K},  s - 1,  2,  (5.3) 

From  this  definition,  it  Is  clear  that 

H(s)  - 9g(u{G’(Xj^,  n);  k > 8,  k e K})  (5.4) 

and 

H(s^)  c 8(82)  for  S2  < Sj^.  (5.5) 

We  suppose  H(p)  i*  , Vp  > N' , p e K for  otherwise  by  choosing  N large  enough, 
G(X|^,  n)  ” ^ N' and  thus  from  (5.2)  we  are  done.  Since  the  sequence 

[Xj^}  Is  convergent,  the  set 

u{G'(Xj^,  n);  k > p,  k e K} 

is  contained  in  a closed  bounded  Euclidean  ball  and  thus  with  g an  LFS  function, 
H(p)  is  the  union  of  a finite  number  of  distinct  subgradient  sets  for  all  p, 
p=l,  2,  .... 

The  sets  of  the  sequence  H(p)  are  nonempty  for  all  p.  Therefore,  there  must 
be  a finite  number  of  subgradient  sets,  say,  r ■ 1,  2,  ...,  q,  each  of  which 
occurs  infinitely  often  In  this  sequence.  In  other  words,  there  exists  some 
N"  such  that 

H(p)  c ufS*^;  r - 1,  ...,  q},  Vp  > N". 

Consequently,  we  can  find  q Index  sets  K^,  r 1,  ...,  q,  where  each  is 
an  infinite  subset  of  K and  S ■ 3g(yj^)  for  some  yj^  c G'(X|^,  n)»  Vk  e K^. 

We  now  show  that  H(p)  5 3g(G'(x^,  n))>  For  fixed  r,  since  the  sequence 
(xj^;  k € K)  is  convergent,  the  sequence  k e K^}  Is  in  a compact  set 
and  thus  has  a limit  point  y^.  Hence  there  exists  a subsequence  c 
such  that  yj^  y^  for  k -*■  »,  k t K\ 
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(5.6) 


Since  Xj^  -*•  x^.  It  easily  follows  that 
£ N(x^,  n). 

Furthermore,  by  Corollary  24.5.1  of  [18]  given  e'  > 0,  > N"  such  that 


38(y^)  5 3g(yj.)  + e'  B,  Vk  c K'^,  k > N^.  (5.7) 

But  » 8g(yj^)  Vk  € K'^  and  thus  (5.7)  clearly  Implies  that 

f igCy^).  (5.8) 

Since  Is  not  a singleton,  from  (5.8)  we  have  that  g Is  not  differentiable  at 
y^,  and  thus  using  (5.6) 

yj.  « G'(x*,  n).  (5.9) 

From  (5.8)  and  (5.9)  It  follows  that  c 3g(G'(x^,  n))  and  hence,  for 
p > N’”  s Max  {N^;  r = 1,  ...,  q}, 

H(p)  c ufS*^;  r = 1,  ...,  q}  c 3g(G'(x*,  n)).  (5.10) 


Now,  since  ag(G'(Xj^,  n))  c H(p)  Vk  £ K,  k > p,  defining  N i Max  {N',  N"'}, 
we  have  from  (5.2)  and  (5.10) 

3g(G’(Xj^,  n))  u 3g(x^)  c 3g(G(x^,  n))  + yB,  k £ K,  k > N, 
or 

3g(G(Xj^,  n))  c 3g(G(x^,  n))  + yB,  Vk  £ K,  k > N.  || 

Lemma  5.3  For  any  e'  >0,  there  exists  a number  such  that  k £ K and 
k > N2  Imply 

S(Xj.,  t,  n)  c S(x*,  e,  n)  + c’B. 

Proof . By  Lemma  5.1,  for  k > N^,  we  have 

S(xj^,  e,  n)  =•  Conv  (ufS^^  (x^,  n);  1 € R(X|jt  e)>) 

c Conv  (u{S^(x^,  n);  1 e R(x*,  e)}). 

For  each  1 £ R(x^,  e),  Sj^(xj^,  n)  was  defined  to  be 

1 


(5.11) 


■■ 


at 


\ 


I 


I 


From  Lenana  5.2,  we  know  that  there  Is  some  number  such  that  for  k £ K, 
k > and  for  all  j » 1,  H, 

8flj(Gij(Xk,  n))  c 3f^j(Gjj(x^,  n))  + e'B/i.  (5.13) 

Thus  (5.12)  and  (5.13)  yield 

S^(Xj^,  n)  c S^(x^,  n)  + e'B,  k e K,  k > N2^,  (5.14) 

for  all  1 € R(x^,  e). 

Let  N2  5 Max  {N2J;  1 e R(x*,  e)}  so  that  k £ K and  k > imply 

u(Si(Xk,  n);  i e R(x*.  e)}  f u{S^(x*,  n);  i s R(x*,  e)}  + e'B.  (5.15) 
Taking  the  convex  hull  of  both  sides  in  (5.15),  from  (5.11)  and  for  k in  K, 
k > N2  H Max  {N^,  Np 

S(Xj^,  e,  n)  c S(x^,  e,  n)  + e'B.  || 

Now,  from  Lemmas  5.1,  5.2  and  5.3,  the  uniform  negativity  of  the  quantity 
i|)(xj^,  e,  n)  is  easy  to  establish. 

Theorem  5.4.  There  exists  a number  N„  such  that  for  k e K and  k > v , the 

^ I 

numbers  t|;(Xj^,  e,  ri)  are  uniformly  negative  and  bounded  away  from  0,  that  is, 
i^(X|^,  e,  n)  £ -b/2  <0.  • 


Proof.  In  the  result  of  Lemma  5.3,  choose  e'  less  than  or  equal  to  b/2.  Then, 

for  any  g with  ) |gj | £ 1 and  k > N2, 

Max  {(d,  g);  d £ S(Xj^,  e,  n)}  < Max  {(d,  g) ; d £ S(x*,  e,  n)  + e'B} 

£ Max  {(d,  g);  d c S(x^,  e,  n) ) + e'. 

Therefore 


i<<(xj^,  e,  n)  - Min  {Max  {(d,  g);  d £ S(Xj^,  e, 
Min  {Max  {(d,  g) ; d £ S(x*,  e, 
1 ^(x*,  e,  n)  + e' 
i iKx*.  e,  n)  + b/2 
< -b/2  . 


n)>;  i |g| I 1 1} 

h)};  I |g|  I 1 1}  + e' 


II 


We  also  need  the  following  three  lemmas: 


20 


Lemma  5.5  CCullum,  et  al. [4])  Let  F be  convex  on  r". 


Let  the  sequences 


and  {gj^}  satisfy 

0 < t < T, 

g*  and  Xj^  -*■  x^,  k -►  ®. 

Then  F'(x^,  g^)  > 0. 

Proof . By  definition, 

F’(x^,  g*)  = lim  [F(x*  + tg*)  - F(x*)]/t. 
t-K)"*" 

But  for  any  t e [0,  T] , 

+ tg*)  - IjL^  F(Xj^  + tgj^) 

i F(x^)  (by  continuity). 

The  desired  result  follows  immediately.  | | 

Lemma  ^.6  There  exists  a number  such  that  for  k > and  k r K,  we  have 
R(x*,  0)  c R(Xj^,  e). 

Proof.  To  prove  this  statement,  we  use  continuity.  Suppose  that  i c R(x^  , 0), 
i.e.,  F(x^)  » f^(x^).  Since  f^  is  continuous,  for  k large  enough,  R ^ 
we  have 

|fi(x^)  - < e/2. 

Thus 

F{x*)  - e/2  < fj(Xj^)  < F(x^)  + e/2. 

Similarly,  since  F is  continuous,  for  k > 

F(x*)  - e/2  < F(x^)  < F(x^)  + c/2. 

Therefore,  k > N3  - Max{MaxfN ;1  e R(x*,  0)},  N’^}  and  k e K imply 
f^(x^)  - F(x^)  > F(x*)  - c/2  - F(x*)  - e/2, 

^ “11  1 « 0)-  Thu*, 

i c R(x^,  c)  , for  all  i c R(x^,  0).  || 
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Lemma  5.7  For  any  e'  > 0,  Vi  £ R(x*,  0)  and  Vs  £ 3f^(x^),  there  exists 
such  that  for  any  k > in  K,  we  can  find  s'  £ S(Xj^,  e,  n)  satisfying 
S=s'+t,  ||t||<E'. 

Proof-  By  Lemma  5.6,  we  know  that  for  k £ K,  k > N^,  R(x^,  0)  c R(x^,  e)  and 


k £ K,  k > NjS^i  £ R(x^^,  e). 

I 

Since  f . = ^ f , by  Theorem  23.8  of  [l8l»  we  can  write  s as 
j=l  ^ 

I 


“ I 8.,  S £ 9f^^(x.). 
j-1  ^ ^ 


(5.16) 


(5.17) 


For  each  j = 1,  2,  ...,  t,  either  f..  is  differentiable  at  x*  or  it  is  not. 

ij 

We  consider  these  as  two  cases. 

A.  f is  differentiable  at  x.,  l.e.,  s,  ■ Vf  (x*)  and  {7f  (x*)}  = 3f..(x*). 

ij  » j ij  " Ij  ij  * 

Then,  according  to  Corollary  24.5.1  of  [18],  for  any  c'  > 0,  there  exists  L^j 
such  that  k c K and  k > imply 


Bflj(Xk)  c 3fj^j(x^)  + e'B/i.  (5.18) 

Since  3f^j(x^)  •>  {Vfj,j(x^)}  - (sj),  (5.18)  means  that  we  can  find 
s'j  c 3flj(Xk)  5 ^^ij^^lj^*k’ 

Sj  - sj  + tj,  lltjll  < e'/i.  (5.19) 

B.  f is  not  differentiable  at  x*.  Then  for  k large  enough  in  K,  k > L.., 
ij  " ij 

X*  € N(Xj^,  n)  and  x^  £ G^j(Xj^,  n).  Hence  for  k>L^j,  if  Sj  £ 3fj^j(x*), 


(5.19) 


Sj  £ 3fij(Gj^j(Xj^,  n)),  l.e.. 


’j  ■ 'j  h'  ‘i  ■ ‘j-  h ■ “• 


(5.20) 


Choosing  now  k > = Max  J “ It  ft  Jl}t  we  obtain,  from  (5.19),  and 


(5.20), 


(5.21) 


where 


£ £ £ £ 

" I 8 - [ (s'  + t ) - J *1  ^ V 

j-1  J J-1  J J j»l  J j.l  J 


S'  £ 3f^j(G  ij(x^t  n)),  lltjll  < c'/t,  J - 1,  2,  ....  1.  (5.22) 
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Defining 


- I t - I t 

j-1  ^ j-1  ■* 


it  is  clear,  from  (5.21)  and  (5.22),  that 

I 


(5.23) 


lltll  - II  I tJ|  < e'. 

J-1  ^ 

Furthermore,  for  k >L^  - Max  (L^,  N^),  (5.16)  Implies  S^(Xj^,  n)  c S(X|^,  e,  n) 
so  that  by  (5.23)  s*  e II 

We  now  possess  the  results  necessary  for  the  main  theorem  of  this  section 
Theorem  5. 8 Any  limit  point  of  a sequence  generated  by  the  algorithm  is 

stationary,  i.e.,  'I'(x*.  h)  - 0. 

Proof . Consider  the  sequence  defined  by 

e,  n)  “ Max  {(d,  g^^);  d c S(x^^,  e,  n)},  k c K. 

Since  {gj^}  c b,  this  sequence  has  a limit  point  g^^  e B.  We  then  have 
x^,  gj^  -►  g^,  for  k -*■  <»  and  k e K*  c K. 

We  know  that  for  some  convex  combination  , 

I - 1.  > 0,  Sj^  € 3f^(x*),  i e R(x*,  0) 

the  directional  derivative  F'(x^,  g^)  is  given  by 
«*)  “ Hd,  g^);  d c 3F(x*)} 

- (I  1^8^,  g^)  (by  (2.4)). 

For  each  1 of  this  convex  combination,  from  Lemma  5.7,  there  exists  such 
that  for  k > Lj  , 

Ve’  > 0,  4 s[  such  that  *4  - 8^  + t^.  8*  c S(x^,  c,  n),  ||t^||  < e’. 

Choosing  k > N,  ■ Max  (L. },  k e K,  we  obtain 
^ 1 ^ 

«*>  • <Z  «*)  (Z  Ijti.  **) 

- (Z  «|j)  + (Z  g*  - g^)  + (^  A^t^.  g^).  (5.24) 
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But, 


(I  1 8j^)5  d c ^'’y  convexity  of  S) 

< n)  (by  definition  of  gj^  and  ij;) 

< -b/2  for  k > N2  (by  Theorem  5.4). 

Thus,  from  (5.24)  and  for  k > Max  {N.,,  N, }, 

2 4 

*^'^***  8*)  i -b/2  + (I  X^s^,  - gj^)  + d Xjtj,  g*).  (5.25) 

Since  gj^  -♦  for  k e K',  there  exists  such  that  k > and  k c K'  imply 

([  8*  - 8^)  1 I II  X^s^l  1 I |g*  - gj^l  I 

< M ||g*  - g^ll  < b/8. 

Furthermore,  choosing  0 < e'  < b/8,  since 

(I  , g*)  < I II  X t I I I |g^| I < e', 

1 1 ^ ^ 

(5.25)  becomes 

F'(x*.  g*)  < -b/4  < 0. 

Considering  now  Lemma  5.5.  with  the  sub-subsequences  (Xj^)  and  (gj^),  k f K* , 

T - + 00,  we  have  the  desired  contradiction.  Thus,  the  assumption  'l>(x*,  e,  n)  < 0 
is  not  valid  and  x^  is  a stationary  point.  | | 
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6.  The  Implemented  Algorithm 


When  actually  executed  on  a computer,  the  result  of  the  difference 
F(x)  - c,  appearing  in  the  definition  of  R(x,  e) , is  not  very  different 
from  F(x)  if  F(x)  is  a large  number  since  In  general  c is  small.  Conse- 
quently, R(x,  e)  might  Indeed  be  reduced  to  R(x,  0)  through  roundoff  error  and 
this.  In  turn,  could  affect  the  convergence  of  the  algorithm  since,  in  effect, 
we  no  longer  consider  e-bindlng  functions. 

To  avoid  this  numerical  problem,  we  redefine  R(x,  c),  as  is  done  in 
[61,  and  use  instead 

R'(x,  e)  « {i  ■ 1,  2,  ...,  m;  f^(x)  ^ F(x)  - e F(x)}. 

It  Is  also  necessary  to  suppose  F*  > 0.  Then,  modifying  in  a straightforward 
manner  Definitions  2.3  and  2.4,  we  have  S*(x,  e,  n)  and  i|<'(x,  e,  n)  replacing 
S(x,  E,  n)  and  i|;(x,  e,  n)  • 

Making  the  obvious  modifications,  most  of  the  previous  results  need  no 
extra  work,  except  for  four  of  them  that  we  consider  now. 

A.  The  inequalities  in  Theorem  3.3  should  be  modified  as 

(1  - e)F(x)  - 2inM  f 1 F(x).  (6.1) 

Hence,  there  is  another  appealing  feature  in  the  use  of  a "relative"  e. 
Rewriting  (6.1)  as 

0 < (F(x)  - F*)/F(x)  < E + 2inM/F(x),  (6.2) 

suppose  that  the  quantity  e + 2inM/F(x)  is  equal  to  10“",  where  n is  some 
positive  Integer.  Then  F(x),  whatever  its  magnitude,  has  at  least  n correct 
digits.  This  is  best  Illustrated  with  an  example: 

F(x)  - 1,000,010  , E + 2inM/F(x)  - lO"^, 

F(x)  - F*  < 10“*F(x) 

< 10.00010  and 
1,000,000.  < F*  < 1,000,010. 
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B.  It  is  easy  to  check  the  correctness  of  the  inclusion 


\ 

I 


R'(x,  e^)  £ R'(x,  Ej)  for  0 £ £2  £ Ej^. 

Then,  we  still  obtain 

R(x,  0)  - R'(x,  0)  £ R'(x,  e),  Ve  > 0. 

Hence,  in  the  remark  following  Example  1 
3F(x)  £ S' (x,  E,  n) 
remains  correct. 

C.  It  is  necessary  to  rederive  Lenmka  5.1,  using  R' (x,  e). 

I.emma  6.1  There  exists  such  that  k £ K and  k > Nj^  imply  R'Cxj^.  t)  £ 

Proof . Using  the  same  argument  as  Dem'yanov  ( 5 1,  it  is  easy  to  establish 
that 

^e',  0 < £ < e*  such  that  R'(x^,  e)  ••  R'(x^,  c'). 

Since  the  f^^  and  F are  continuous,  we  have 

such  that  k e K and  k > Nj^  imply 
|fl(Xk^  - fjCx*)!  < (e*  - e)  F(x*)/2,  1 - 1,  2,  ....  m,  and 

such  that  k € K and  k > l«ply 
|F(x^)  - F(x^)|  < (E*  - e)F(x*)/2(1  - e). 

Thus  for  k £ K and  k > N ■ Max  {N|^, 

F(x*)  - (e'  - e)F(x*)/2(1  - e)  < F(Xj^),  (6.3) 

ffCXk)  < Fj(x^)  + (e’  - e)F(x*)/2.  (6.4) 

Multiplying  (6.3)  by  1 - c and  considering  also  (6.4),  we  obtain 
(1  - e)F(x*)  - (e’  - e)F(x*)/2  < (1  - c)F(x^) 

< fjCx^)  (if  1 c R'(x^,  c)) 

< fj(x*)  + (c'  - €)F(x*)/2, 

that  is, 

(1  - e')F(x^)  < f^(x*)  VI  £ R’CXj^t  c).  (6.5) 
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But  (6.5)  means  that  Vi  e R'(x^.  e),  i € R'(x^,  e').  Since  R’(x*,  e')  - 
R'(x*.  c),  for  k > Nj,we  have  R'(Xj^,  e)  £ R' (x^,  c).  || 

D_.  Lemma  5.6  needs  also  to  be  modified  as  follows: 

Lenrna  6.2  There  exists  such  that  for  k c K and  k > N^,  we  have 
R(x*,  0)  - R'(x*,  0)  c R'Cxj^,  e). 

Pr^.  Choose  any  i e R’(x^,  0),  so  that  fj(x*)  - F(x*)  - F*.  Since  f^  is 
continuous,  there  exists  such  that  for  k e K and  k > 

IfjCx*)  - fi(Xj^)|  < eF*/2.  (6.6) 

Similarly,  with  F,  3n  ^ such  that  for  k > N^, 

|f(x*)  - F(xj^)|  < eF^/2.  (6.7) 

Hence,  for  k > N3  = Max{Max{N*^;  1 e R'(x^,  0)},  N^}.  we  get  from  (6.6)  and  (6.7), 
fi(xj^)  > fi(x*)  - e F*/2 

> F*  - E F*/2,  and  (6.8) 

-F(xj^)  > -F(x^)  - e F*/2 

> -F*  - e F*/2.  (6.9) 

Adding  (6.8)  and  (6.9)  yields 

fi<Xk)  - F(Xj^)  > - E F* 

> - e F(xj^)  (since  F(xj^)  > F*)  , 

that  is, 

i € R’  (Xj^,  e).  II 

These  are  the  four  principal  modifications  to  signal  and,  mutatis  mutandis, 
the  convergence  proof  remains  valid. 
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2^ Compj^t atlon  Results 


We  have  Implemented  our  algorithm  with  the  modification  given  in  Section  6. 

To  find  the  point  of  minimum  norm  in  S(xj^,  e,  n),  we  used,  depending  on 
the  problem  considered,  two  different  algorithms.  For  the  minlmax  problems 
with  m > 1 and  t - 1,  we  used  an  algorithm  of  Wolfe  [20].  Otherwise,  for 
mlnisum  problems,  where  m • 1 and  i > 1,  we  employed  an  algorithm  from  Gilbert  [10]. 

The  line  search  was  done  with  quadratic  fits  and  worked  well  for  most 
problems.  We  modified  it  when  dealing  with  piecewise  linear  functions,  since 
in  this  case  a quadratic  fit  does  not  seem  reasonable,  and  devised  a different 
line  search  which  has  performed  satisfactorily. 

We  solved  three  types  of  problems;  minlmax  location,  mlnisum  location 
and  approximation  problems.  Description  of  the  results  are  given  below. 

Minlmax  location  problems . 

The  problem  to  solve  is 

Min  Max  (f/x);  1 - 1,  2 m} 

x«R 

where  f^(x)  may  have  one  of  the  following  expressions 

”i  I l^k  “ ®ll  lp»  fo*"  sojne  Pf  1 < p < <», 

'*1  " ’'j  Np.  for  some  p,  1 < p < », 

and  X ■ (Xj^,  . . . , ^x^,  . . . , x^)  eR,  Xj^eR®,  k“l,  ...,r  and  n •«  sr. 

The  a^  are  Interpreted  as  known  locations  in  R®  of  existing  facilities  and  the  Xj^  are 
the  locations  of  the  new  facilities,  and  where  the  w^^  are  positive  weights. 

Results  of  the  first  minlmax  location  problems  solved  are  given  in  Table  1. 

M Indicates  the  number  of  existing  facilities  and  N the  nuiid>er  of  new  facilities 
to  be  located.  We  ran  three  Type  1 problems  involving  t^'oorms,  which  were 
randomly  generated  with  weights  between  1 and  100  and  existing  facility  locations 
in  the  square  1000  by  1000.  Problem  2 was  a selected  minlmax  location  problem 


posed  by  Love,  Wesolowsky  and  KVaemer  and  considered  in  [7].  Problem  2 involves 
i2*“norms.  Problems  3 and  4 are  problems  discussed  in  [7].  The  data  in 

Problem  5 is  identical  to  the  data  in  Problem  2,  however  -norms  are  used. 
Finally,  Problem  6 is  a randomly  generated  problem  using  t^-norms. 

For  comparative  purposes,  we  have  Included  the  execution  times  on 
these  problems  using: 

(a)  The  heuristic  subgradient  procedure  of  Hearn  and  Lowe  [12]  (all  problems). 

(b)  A dual  formulation  solved  by  GRG  [7 ] » in  the  case  of  1. -norms  (Problems 
1 through  4) . 

(c)  Another  dual  procedure  studied  by  Dearlng  and  Francis  [5],  in  the  case 
of  Ij^-norms  (Problems  5 and  6). 

(d)  The  subgradient  algorithm  (all  problems). 

We  have  also  listed  the  number  of  iterations  using  the  subgradient  algorithm. 
In  those  cases  where  the  last  iterate  was  not  a stationary  point,  the  line 
search  could  not  generate  a point  to  Improve  the  objective  function  value. 


Problem 


Execution  Time 


Iterations 


(a) 

(b) 

(c) 

(d) 

1) 

Random  data,  )l2“norm 

1.66s 

2.03s 

— 

3.558 

25 

M = 50,  N -1 

1.15s 

1.578 

— 

1.528 

11* 

.95s 

2.288 

— 

2.52s 

16 

2) 

Love  et  al,  l.-norm 

M - 5,  N - 2 ^ 

.39s 

2.06s 

— 

.66s 

14* 

3) 

Triangle  #1,  l.-norm 

M - 3,  N - 3 ^ 

.64s 

5.58 

— 

l.lls 

16* 

Triangle  #2, 

M - 3,  N - 3 

.808 

10.58 

— 

1.40s 

20* 

5) 

Love  et  al,  l^-norm 

M - 5,  N - 2 ^ 

.24s 

— 

2.408 

.178 

6* 

6) 

Random  data,  l.-norm 

1.168** 

— 

2.898 

6.948 

15 

* The  last  iterate  is  a stationary  point 
**  Used  a fixed  step  length. 
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For  the  problems  of  Table  1,  the  value  chosen  for  c was  5x10  ^ and  we 
set  n “ 10  ^ or  10 

We  also  devised  a problem  of  our  own  which  involved  mixed  norms.  The 
problem  scenario  reads  as  follows.  Two  ships  have  to  be  located  in  the  Caribbean 
sea  and  must  be  ready  to  intervene,  in  case  of  trouble,  at  any  one  of  nine  given 
cities  of  the  Caribbean  Islands.  Trouble  may  occur  according  to  estimated 
probabilities  which  are  used  as  weights.  Furthermore,  the  two  ships  must  be 
able  to  communicate  and  we  must  consider  their  mutual  distance  (Euclidean  distance 
with  weight  1).  Thus,  the  problem  is 


Min 

X 6R 

^ 2 
x^cR 


Max 

i-1.. 


.,9 


{w^j^l  |xj 


l|}. 


The  cities  and  their  locations  are  displayed  in  Table  2.  For  some  cities, 

the  distance  to  a ship  Is  well  represented  as  Euclidean  distance.  But  if  one 

city  is  on  the  opposite  side  of  the  island  with  respect  to  the  ship  location, 

we  can  no  longer  use  Euclidean  distance  but  an  I -distance  where  p is  chosen 

P 

between  I and  2 (see  Table  2). 


cities 

locations 

1st 

ship 

2nd 

ship 

''ll 

Pll 

”12 

Pi2 

Colon  (Panama  Canal) 

11.4, 

11.6 

2.0 

2.0 

1.0 

2.0 

Caracas-LaGuaira  (Venezuela) 

35.3, 

13.5 

1.0 

2.0 

2.0 

2.0 

Havana  (Cuba) 

8.80 

, 37.2 

1.5 

1.1 

1.0 

1.4 

Guantanamo  (Cuba) 

20.9, 

30.6 

1.5 

1.5 

1.0 

1.9 

Port-au-Prince  (Haiti) 

25.5, 

28.0 

1.5 

1.4 

1.5 

1.2 

Santo  Domingo  (Dorn.  Rep.) 

29.7, 

27.7 

1.0 

2.0 

1.5 

2.0 

San  Juan  (Puerto  Rico) 

36.2, 

27.8 

0.5 

1.8 

1.0 

1.7 

Fort-de-France  (Martinique) 

45.5, 

21.3 

0.5 

2.0 

0.5 

2.0 

Montego  Bay  (Jamaica) 

15.8, 

28.2 

0.5 

1.1 

0.5 

1.8 

Table 
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The  optimal  locations  are  (13.817,  24.358)^  and  (25.818,  22.454)^,  with 

F*  = 26.0836.  This  solution  was  found  after  40  iterations  and  3.15s  of  CPU 

time.  As  a final  check,  we  used  the  locations  for  the  ships  given  by  the 

algorithm  and  measured  the  l~  distances  between  the  ships  and  ports  and  the 

*-p“dlstance  between  the  ships.  All  Z.p-dlstances  approximately  agreed  with 

the  t -distances  used  in  the  algorithm  with  one  exception.  The  t -distances 
^ P 

from  the  ships  to  Port-au-Prince  were  not  in  agreement  with  the  Up-distances 
used  in  the  algorithm,  but  this  was  not  Important  in  our 
problem  since  at  the  optimal  solution,  the  functions  involving  Port-au-Prince 
were  not  binding. 

Minisum  location  problems. 

The  objective  function  to  minimize  has  the  form 

F(x)  =•  f (x) , X f r”, 
i 

where  the  f^  are  as  in  the  minlmax  location  problem. 

Among  many  minlsum  problems  that  we  solved,  we  give  here  the  numerical 
results  for  two  problems: 

a)  The  24  cities  problem  of  Kuhn  and  Kuenne  [13]. 

Starting  from  the  center  of  gravity  of  the  24  existing  facilities, 

* 

Xq  - (47.47945,  34.35616),  the  algorithm  generates  the  result  x ■ (47.70779, 
35.10391)  in  6 iterations  and  .46s  of  CPU  time,  giving  F*  • .9528840. 

After  5 iterations,  the  procedure  of  Kuhn  and  Kuenne  reaches  the  result 
(47.60,  35.32),  which  corresponds  to  an  objective  function  value  of  .9541373. 

b)  A problem  from  Eyster  et  al.  (8  1,  with  five  existing  and  two  new 


facilities. 


The  starting  point  is  Xq  “ (0,  0,  0,  0)  and  after  23  iterations  we  got 
X*  » (2.840055,  2.686639,  5.129290,  6.388482)  with  F*  = 67.23856.  The  CPU 
time  for  this  problem  is  1.07s.  The  HAP  procedure  used  in  [8],  starting  also 
at  Xq,  stopped  after  45  iterations  and  gave  x*  = (2.840,  2.687,  5.126,  6.383). 
F*  = 67.239 

In  both  problems  a and  b,  we  chose  n “ 10 
l.inear  Approximation  Problems 

A linear  approximation  problem  can  be  formulated  as  follows 

Min  K(Ax  - b),  where  A is  an  m x n matrix  and  b is  a vector  of  R™ 
xeR 

and  where  K is  some  norm  in  R™  measuring  the  discrepancy  between  a desired 
point,  b,  and  an  approximation.  Ax,  of  this  point. 


Popular  choices  for  K are  the  tj^-norm  and  the  i^-noni\  so  that  we  solve  either 
m 

Min  I |(Ax)  - b I, 

xeR"  1=1  ^ ^ 


or 

Min  Max  { | (Ax)  - b |;  1-1,  2,  ...,  m}. 
xeR  ^ ^ 

With  the  i^-norm,  and  thus  a mlnimax  problem,  we  solved  the  examples  given 
by  Barrodale  and  Young  [ i,  p.  115].  For  these  problems,  we  chose  e-io"^  and  n-io"^^. 
The  results  are  presented  In  Table  3.  The  solutions  we  found  are  in  total 
agreement  with  those  of  ( 1 ] . Barrodale  and  Young  did  not  report  any  computation 
times  or  numbers  of  iterations. 


Problem 

Execution  Time 

Iterations 

Optimal  Value 

1) 

Example  1 
m - 33,  n - 3 

.85s 

12* 

.6733  X 10~^ 

2) 

Example  2 
m » 41,  n - 6 

9.90s 

58* 

.1514  X 10'^ 

3) 

Example  3,  Case  1 
m - 51,  n - 5 ^ 

2.54s 

41* 

.1039  X 10~^ 

4) 

Example  3,  Case  2 
m - 51,  n - 5 

2.86s 

39* 

.1757  X lO"^ 

* The  last  iterate  la  a stationary  point. 

Table  3 
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